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ABSTRACT 

This  thesis  is  a  case  study  that  compares  actual 
costs  to  estimated  costs  for  a  State-of-the-Art  ( SOA ) 
extension.  The  Advanced  Nuclear  Gamma-Ray  Spectrometer 
(ANGAS)  program  initiated  by  the  Defense  Advanced 
Research  Project  Office  (DARPA)  in  conjunction  with  the 
Research  and  Development  Division  of  Lockheed  Missiles 
and  Space  Company  (LMSC),  Inc.  is  the  subject  of  this 
case  study. 

This  thesis  identifies:  the  original  description  of 
the  technology  extension;  the  methods  used  at  Lockheed 
to  develop  SOA  extension  cost  estimates;  specific 
factors  that  helped  and  hindered  accurate  cost 
estimating  and  significant  variances  in  cost  and 
technological  progress. 
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I .     INTRODUCTION 

A.  BACKGROUND 

The  development  of  new  weapon  systems  for  the 
Department  of  Defense  ( DOD )  frequently  requires  State 
of-the-Art  ( SOA )  extensions.  The  difficulty  of 
estimating  development  and  early  production  costs  for 
new  systems  when  SOA  extensions  are  involved  often 
leads  to  cost  overruns.  In  fact,  contractors'  costs 
exceed  project  manager's  estimates  for  new  projects  60 
percent  of  the  time.  [REF.  1  :  p.  50-51.] 

Prior  research  has  not  solved  the  problem  of 
accurately  estimating  SOA  extension  costs.  These 
development  and  production  cost  overruns  make  it 
difficult  for  DOD  to  accurately  compare  existing  weapon 
technology  with  undeveloped  technology. 

Professor  Willis  R.  Greer,  Jr.,  Naval  Postgraduate 
School,  Monterey,  California,  in  conjunction  with  the 
Naval  Sea  Systems  Command  (NAVSEA),  is  attempting  to 
develop  more  reliable  methodology  for  estimating  the 
cost  of  extensions  as  applied  to  electronic  systems. 
This  thesis  provides  data  to  support  that  effort 
through  the  case  study  of  an  SOA  extension. 

B.  OBJECTIVES 

This  thesis  details  the  Advanced  Nuclear  Gamma-Ray 
Spectrometer  (ANGAS)  experiment  program  initiated  by 
the  Defense  Advanced  Research  Projects  Office  in 
conjunction  with  the  Research  and  Development  Division 
of  Lockheed  Missiles  and  Space  Company,  Inc.  The 
objectives  are  outlined  below: 

1.  Identify  the   original  description  of  the 
technology  extension, 

2.  Document  estimated  development  costs, 


3.  Describe  the  methods  used  to  develop   the  cost 
estimates , 

4.  Compare  actual  cost  data  to  estimated  costs, 

5.  Analyze  and  measure  any  significant  variances 
in  cost  or  progress  in  technological 
development , 

6.  Identify  specific   factors  that  help  or  hinder 
accurate  cost  estimating. 


C.  RESEARCH  QUESTIONS 
Primary  Research  Question: 

How  do   estimated  development   and  production  costs 
compare  with  actual  costs  for  the  ANGAS  project? 
Subsidiary  Research  Questions: 

1 .  What  methods  and  techniques  were  used  to 
estimate  development  and  production  costs? 

2.  How  was  technological  progress  measured  and 
evaluated? 

3.  What  factors  helped  or  hindered  accurate  cost 
estimating  and  accomplishment  of  stated  SOA 
extension  goals? 

D.  SCOPE  LIMITATIONS  AND  ASSUMPTIONS 

This  thesis  is  a  case  study  of  the  ANGAS  project. 
The  primary  task  is  to  obtain  and  analyze  estimated  and 
actual  cost  data,  document  cost  estimation  procedures 
for  the  ANGAS  project  and  to  evaluate  generic  cost 
estimating  procedures  used  by  Lockheed  for  SOA 
projects.  The  technological  requirements  of  the  ANGAS 
project  are  studied  to  measure  the  accomplishment  of 
stated  ANGAS  technological  goals. 

The  ANGAS  project,  an  ongoing  effort  at  Lockheed, 
is  designed  to  produce  one  end  unit  to  fly  aboard  a 
satellite.  Thus,  the  cost  data  and  estimation 
procedures  presented  in  this  paper  are  more  applicable 
to  development  costs  rather  than  to  production  costs. 
This  is  important  because  many  SOA  extensions,  by  their 


nature,   must   be   implemented   through   a   substantial 

development   effort.      At   the   outset   of   many   SOA 

development   efforts,   the   end   unit   is   more   of   an 

abstract  idea  than  a  detailed  production  specification. 

In  the  1970 's,  system  acquisitions  programs  were 
usually  focused  on  specific  technical  approaches  at 
the  time  of  their  initiation. 

Today  a  program  is  initiated  after  competent 
authority,  the  Program  Decision  Authority,  approves  a 
specific.  formally  stated  mission  need,  based  on 
Mission  Area  Analysis,  submitted  within  the  first 
Program  Objectives  Memorandum  in  which  program  funds 
are  requested.  [REF.  2  :  p.  1-7. J 

E.   METHODOLOGY 

The  methodology  used  in  this  research  is  case 
study.  Research  trips  to  Lockheed  included  visits  with 
the  ANGAS  contract  administrator,  principle 
investigator,  senior  cost  estimator  and  several 
development  engineers.  These  Lockheed  personnel  are 
located  at  facilities  in  Sunnyvale  and  Palo  Alto, 
California.  Lockheed  personnel  provided  estimated 
costs,  the  associated  cost  estimation  methods,  actual 
costs  incurred  to  date,  a  summary  of  progress, 
technological  breakthroughs  and  specific  factors  that 
have  helped  and  hindered  the  SOA  extension  effort. 

The  NAVPRO  office,  located  in  Sunnyvale,  was 
visited  numerous  times  to  identify  a  suitable  contract 
for  study  and  to  obtain  relevant  contract  and  cost 
data . 

The  ANGAS  program  manager  was  also  interviewed  to 
determine  what  the  government  hoped  to  achieve  from  the 
ANGAS  project  and  how  technological  progress  of  that 
project  is  measured. 

Background  and  research  data  were  obtained  through 
the  Naval  Postgraduate  School  library  and  its  research 
services.  Automated  searches  were  conducted  through 
the  NPS  library's  DOD ,  NON-DOD  and  Defense  Technical 
Information   Center's   files.      Information   obtained 
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through  visits   to  Lockheed   during  the   case  study  and 
related  research  material  is  contained  in  this  thesis. 

F.   SUMMARY  OF  FINDINGS 

Major  findings  of  this  thesis  are  summarized  below: 

1.  There  were  no  significant  variances  between 
estimated  an  actual  costs  to  date.   Spending 
was  at  a  slightly  lower  rate  than  anticipated. 

2.  Like  many  research  and  development  projects 
that  are  administered  by  a  "level  of  effort 
contract",  measurement  and  analysis  of 
technological  development  is  the  primary  gauge 
of  effectiveness  and  progress. 

3.  SOA  extensions,  achieved  by  numerous  smaller 
scale  projects  that  incrementally  extend  the 
SOA,  reduce  the  risk  of  both  costs  overruns  and 
failure  to  achieve  technological  goals. 

4.  Personnel  costs  are  the  most  significant  cost 
factor  in  the  development  phase  of  SOA 
extensions . 

5.  Incremental  funding  of  a  DOD  SOA  project 
provides  leverage  to  the  government  that  can 
reduce  cost  overruns  and  achieve  desired 
technological  goals  but  tends  to  increase  the 
length  of  time  required  for  the  development 
effort . 

6.  Currently  available  parametric  models,  used  in 
private  industry,  could  be  applied  to 
government  use  with  proper  historical  data 
application  and  case-by-case  analysis. 


G.   ORGANIZATION  OF  STUDY 

Chapter  II  provides  a  background  of  the  ANGAS 
project,  an  overview  of  cost  estimation  procedures  used 
for  ANGAS  and  a  summary  of  cost  estimation  procedures 
and  models  used  at  Lockheed. 

Chapter  III  details  the  methodology  used  in  this 
thesis  and  present  data. 

Chapter  IV  analyzes  the  data  in  chapter  III 
consistent  with  the  stated  objectives. 

Chapter  V  states  the  conclusions  drawn  from  this 
SOA  extension  case  study. 


II .  BACKGROUND 

A.   PROJECT  OVERVIEW 

The  Defense  Advanced  Research  Projects  Agency  -  201 
ANGAS  program  involves  the  design,  test  and 
fabrication  of  the  ANGAS  flight  instrument  and  the 
performance  of  a  long  term  flight  demonstration  in 
space.   [REF.  3  :  p.  3-2. J 

The  ANGAS  project  is  designed  to  develop  a  gamma- 
ray  spectrometer  that  will  be  carried  into  orbit  by 
satillite.  The  spectrometer  is  designed  to  pasively 
detect  radiation  in  space  through  an  array  of  germanium 
sensors.  The  project  will  be  completed  when  the 
designed  spectrometer  has  been  flown  in  space  and  the 
specified  scientific  results  have  been  supplied  to  the 
government.  The  specific  requirements  of  the  ANGAS 
project  are  provided  in  the  Statements  of  Work  listed 
in  Appendix  B. 

The  ANGAS  system  required  development  of  a  large 
multi-element  array  of  new,  n-type,  segmented  germanium 
sensors.  The  design  concepts  represent  a  research  and 
development  effort  which  has  never  been  performed 
before.  This  SOA  extension  in  gamma-ray  spectrometers 
incorporates  the  following  new  technologies: 

1.  Imaging  collimator, 

2.  Cooled,  radiation  resistant  n-type  germanium 
sensors , 

3.  Anticoincidence  shield, 

4.  Electronics  subsystem  which  enables  rapid 
onboard  data  analysis  and  microprocessor 
control , 

5.  Solid  cryogen  cooler. 

The  ANGAS  program  is  being  developed  by  Lockheed 
Space  Sciences  Laboratory  which  is  part  of  the  Lockheed 
Research    and    Development   Division,   the   principle 
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research   organization   of   the   Lockheed   Missiles  and 

Space  Corporation. 

The  government's  program  manager  for  ANGAS  is  LTCOL 

George   P.   Lasche,   USA.   He   is   in   the   Geophysical 

Sciences   division   at   the   Defense   Advanced  Research 

Projects  Office,  Arlington,  VA.   He  described  the  ANGAS 

program  goals  as: 

An  effort  to  develop  new  techniques  and  improve 
sensitivity  and  capability  to  record  and  passively 
detect  radiation  in  space  through  1a  process  of 
imaging  with  high   energy  resolution.1 

A   wide   variety   of   potential   DOD  uses  for  ANGAS 

include  its   use   as   a   means   to   verify   arms  treaty 

compliance  from  space. 

B.   OVERVIEW  OF   COST  ESTIMATION  PROCEDURES  USED  FOR 

ANGAS  CONTRACT  PROPOSALS 

The  principle  method  used  by  Lockheed  Space 
Sciences  Laboratory  to  determine  the  cost  estimates  for 
ANGAS  contract  proposals  is  a  bottom-up,  engineering 
estimate.  To  date  approximately  $4.96  million  has  been 
funded  in  two  increments,  in  a  "level  of  effort"  type 
contract.  The  projected  total  funding  for  the  entire 
ANGAS  project  will  be  approximately  17  million  dollars. 

Dr.     George     Nakano ,     Lockheed's     principle 

investigator  (project  director  /   technical  supervisor) 

for  ANGAS  explained  that  the  ANGAS  cost  estimate: 

is  a  bottom-up  estimate  that  is  developed  based  on 
past  experience  of  costs  ^incurred  developing  similar 
satellite  borne   systems/ 

The    bottom-up    estimating   methodology   used   at 

Lockheed  provides  detailed  functional   and  cost  element 

estimates  provided  by  the  lowest  competent  level  in  the 

organization.  The   cost   estimates   developed   for  each 


1  Information     was     obtained     by    telephone 
conversation  on  September  3,  1987. 

2  Information  was  obtained  in  an  interview  at  Palo 
Alto,  CA  on  September  4,  1987. 
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phase  of  the  ANGAS  contract  were  based  on  historical 
cost  data,  the  requirements  of  the  Work  Breakdown 
Structure  (WBS)  and  the  Statement  of  Work  (SOW)  items 
listed  in  the  Requests  for  Proposal  (RFP). 

Lockheed's  contract  pricing  proposal  for  ANGAS 
contained  the  following  summarized  items.  [REF.  3  :  p. 
2.2-2.3] 

1.  The  overhead,  general  and  administrative  rates 
are  applied  as  a  percent  times  direct  labor 
dollars.  The  rates  were  forecasts  based  on 
historical  data  and  current  projections. 

2.  Material  requirements  are  identified,  listed 
and  priced  by  component  or  subsystem.  These 
direct  purchase  items  range  from  firm  prices  to 
estimates  based  on  the  accuracy  that  program 
requirements  were  defined. 

3.  Procurement  burden  is  applied  as  a  percent  of 
the  total  estimated  costs  for  material, 
purchased  serices  and  subcontract  and  outside 
production  costs.  The  rates  are  based  on 
historical  data  and  current  projections. 

4.  Cost  of  money  is  an  element  of  facilities 
capital.  The  rates  are  applied  to  the  direct 
labor  dollars  associated  with  annual  overhead 
and  general  andadministrat ive  rates  and  to  the 
base  amount  associated  with  the  annual 
procurement  burden.  The  cost  of  money  rate  used 
was  10.375  percent. 

5.  Air  fare  rates  are  averages  of  recent  Lockheed 
ticket  costs  for  tourist  class  to  the 
designated  cities.  Per  diem  rates  are  based  on 
historical  data  and  current  projections. 

6.  Direct  labor  hours,  the  most  significant 
component  of  the  ANGAS  cost  estimate,  are  based 
on   a   technical   evaluation   developed   from  a 
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detailed  analysis  of  the   program  requirements. 

The  basic  labor  rates  are  current  average  rates 

for  direct  labor  pools,   by   skill    categories 

that  are   used  in   the  program.  These  rates  are 

escalated  by  approved  factors   when  appropriate 

to  allow  for  future  labor  rate  increases  during 

the  program. 

All  costs  and  applied   rates   were   audited   by  the 

Defense  Contract   Audit  Agency   and  negotiated  with  the 

cognizant  government   activity.   Specific   costs,  rates 

and  audit  results  will  be  discussed  in  Chapters  III  and 

IV. 

Labor  hour  and  cost  estimates  were  based  on 
previous  satellite  flight  programs  involving  similar 
but  much  smaller  gamma-ray  sensor  systems  developed  by 
Lockheed.  The  Defense  Advanced  Research  Projects  Agency 
301  program  although  only  one-forth  the  size  of  ANGAS 
and  less  technically  complex  was  used  in  developing  the 
ANGAS  cost  estimate. 

C.   SUMMARY  OF  COST  ESTIMATION  PROCEDURES  AND  MODELS 

USED  BY  LOCKHEED 

The  LMSC  Estimating  Systems  Description  ( ESD  ) 
Manual  provides  company-wide  guidance  for  estimating 
requirements  for  government  or  prime  customer 
contracts . 

The  LMSC  ESD  Manual  states: 

overall  responsibility  for  the  LMSC  estimating 
definition,  development,  control  and  compliance  is 
vested  in  the  Vice-President,  Finance.  The  Division 
Industrial  Accountants  report  to  the  Vice-President 
Finance  and  are  responsible  for  ensuring  compliance 
with  this  document.  [REF .  4  :  p  1.3] 

The  following  summary  of  cost  estimation  procedures 

and  models  is  provided  from  information  obtained  from  a 

presentation  on  SOA  pricing  by  Mr.   Ted  Castro,  Manager 

LMSC  Estimating   Systems,  Mr.  Donald  H.  Palmby,  Manager 

LMSC  Cost   Modeling  and   Analysis,  and   Mr.  Ken  Peeler, 
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Chief  Industrial   Accountant  LMSC  Astronautics  Division 
and  the  LMSC  ESD  Manual . 

1 .   SOA  Proposal  Estimating  Process 

Figure  2-1,  which  follows  this  description  of 
Lockheed's  contract  proposal  estimating  procedure 
provides  a  flow  diagram  of  the  process . [REF .  4  :  p. 
2.32  -  2.62] 

The  cost  kick-off  meeting  establishes  a 
schedule  of  deadlines  for  completion  of  cost  inputs. 
Cost  inputs  and  bases  are  generated  for  materials, 
direct  labor  and  other  direct  costs. 

The   Federal    Acquisition   Requirements   (FAR) 

directs  DOD  contractors  to: 

provide  a   consolidated  price   summary  of  individual 
material  quantities  included  in  various  tasks,  orders 
or  contract   line  items  being  proposed  and  the  basis 
for  pricing  (vendor   quotes,   invoice   prices,  etc.). 
[REF  5.  :  15.804-6] 

Material   requirements   are   broken   down   into 

three  categories: 

1.  Subcontract  estimates:  obtained  through  formal, 
detailed  RFPs  or  through  estimation  if 
sufficient  time  is  not  available  to  identify  a 
subcontractor  and  obtain  a  price  quote  before 
Lockheed  must  respond  to  the  government 
proposal.  Lockheed  is  required  to  establish  the 
reasonableness  of  any  estimate  through  an 
independent  cost  analysis  or  technical  basis. 

2.  Material  estimates:  includes  prices  for 
standard  commercial  items,  raw  materials, 
purchased  parts  and  material  usage  costs.  These 
material  items  are  priced  through  historical 
data,  supplier  quotations  and  estimations. 

3.  Intra-Lockheed  Work  Transfer  ( IWT )  Items:  these 
material  items  are  made  at  Lockheed.  FAR 
requires   that   these   items   be  transferred  at 
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cost  with  an  explanation  of  the   pricing  method 

used . 

Once  all  material  requirement  costs  are 
consolidated  they  are  reviewed  and  additional  analysis 
and  justification  is  provided  if  needed  or  required  by 
FAR.  A  consolidated  price  summary  of  all  material 
requirements  is  then  generated.  Lockheed's  final  action 
for  material  costs  is  the  application  of  material 
escalation  rates  if  justified  for  a  particular  contract 
or  contract  phase. 

Direct  Labor  cost  estimates  must: 

provide  a  time-phased  (e.g.  monthly,  quarter ly , etc .  ) 
breakdown  of  labor  hours,  rates  and  cost  by 
appropriate  category,  and  furnish  bases  for 
estimates.  [REF.  5  :  FAR  15.804-6] 

Direct  labor  estimates  are  usually  expressed  by 

Lockheed   in    labor   hours,    identified   directly   to 

research,    design,    production    or    other    project 

objective.   To   justify   direct   labor   estimates,   the 

requirements  of   an  SOA   extension  RFP   are  broken  down 

into  tasks   consistent   with   SOW   requirements   or  WBS 

element.  The   two  basic  methods  used  to  provide  a  basis 

of  estimate  (BOE)  for  hours  quoted  are: 

1.  Historical  or  similar  to  data:  this  is  the 
preferred  BOE  for  quoting  direct  labor  hours. 
Similar  programs  are  used  to  identify  tasks 
that  are  equivalent  to  tasks  to  be  performed  in 
the  proposed  contract.  Adjustments  or  applied 
factors  are  often  used  to  calibrate  historical 
data.  Justification  of  adjustments  or  factors 
is  required  by  FAR. 

2.  Engineering  estimates:  used  when  no  historical 
or  similar  to  data  is  available.  Tasks  are 
broken  down  to  the  lowest  level  of  detail. 

Use  of  estimating  standards  is  a  direct  labor 
estimating  method  which  relates  production  costs  to 
specific  characteristics  of  a  product,  such  as  weight, 
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size  or  composition. 

Cost     estimating     standards  must    be    applied 

consistently  based  on  verified.  uniformly  correlated 

data  expressed   in  the  same  unit  of  measure . [REF .  4  : 
p.  2.60] 

A  learning  curve,  which  predicts  the   extent  to 

which  experience  in  performing  a  task  will  decrease  the 

labor  time  needed  to  perform  that  task,  is  then  applied 

by  Lockheed  to  all  labor  estimates.  The  LMSC  labor  cost 

accumulation  system   is  the   data  base   used  to  develop 

learning  curves.    Curve  selection  and  slope  definition 

depend  on  a  number  of  factors,  including: 

1.  how  the  data  has  been  accumulated  (by  lot   or 
un  it), 

2.  whether  the  item  is  a  follow-on  to  an  existing 
production  run  or  an  estimate  based  on  a 
similar  item, 

3.  if  a  production  break,  change  in  design 
technology  or  factory  method  has  occurred,  and 

4.  customer  specification  of  the  methodology, 
[REF.  4  :  p  2.61] 

Once   total   labor   hour  estimates  are  compiled 

and  reviewed    Lockheed   direct   labor   hour   rates  are 

applied  to   the  labor   hours.  The   estimation  of  "other 

direct  costs"  and  BOE  is  the  final  major   cost  category 

that  must   be  derived  for  an  SOA  extension  price  quote. 

Other  direct  costs  include: 

travel,  overtime  premiums,  allocated  prime  cost 
(APC)/pooled  work  orders,  monitored  line  parts  APC , 
relocation  expenses  and  foreign  field  service 
bonuses . [REF.  6  :  p.  2-62J. 

Other  direct    costs   are    usually   based   on 

historical   or  direct    estimating    procedures.   All 

estimates  for  materials,  direct  labor  and  other  direct 

costs  form  the  initial   cost   proposal   which   is    now 

reviewed   and  approved   or   disapproved.   If   approved 

management  price  review  and  negotiation    and  sometimes 

best  and  final  offer  steps  follow. 


16 


2 .    Bottom  -  Up  Cost  Estimating  Method 

Lockheed  uses  a  multiple  estimating  approach. 
When  detailed  task  definition  is  available  a  bottom  - 
up  /  engineering  estimate  is  prepared  by  the  cognizant 
division(s).  These  bottom  -  up  estimates  provide: 

1.  High  degree  of  accuracy  and  traceabi 1 i ty , 

2.  Cost  breakdowns  to  the  lowest  levels  of  the 
WBS, 

3.  Based  on  historical  data. 

The   primary   disadvantages   of   a   bottom   -   up   cost 
estimate  for  an  SOA  extension  effort  are: 

1.  Dependence  on   detailed  design  specifications 
which  do  not  usually  exist  during   early  phases 
of  SOA  extension  projects, 

2.  Expensive  and  time  consuming, 

3.  Cannot  be  used  as  the  sole  costing  method 

4.  Trade  -  off  analysis  is  not  easily  evaluated. 

Although  many  SOA  extensions  do  not  initially 
provide  detailed  design  specifications,  bottom  -  up 
cost  estimating  is  sometimes  still  used  by  Lockheed. 
Estimates  are  based  on  the  experience  gained  by 
individual  engineers  and  technicians  from  developing 
systems  or  components  that  used  existing  SOA 
technology.  Many  of  the  SOA  extension  contracts  awarded 
to  Lockheed  are  an  extension  of  technologies  and 
systems  developed  previously  by  Lockheed.  Bottom  -  up 
SOA  extension  cost  estimates  are  reconciled  with 
experience  and  computer  cost  models. 

Various  computer  models  are  used  in  SOA  cost 
estimating  by  Lockheed  to  establish  "price  targets", 
provide  "sanity  checks"  and  reconcile  other  cost 
estimations . 

3 .   Parametric  Cost  Models 

Lockheed  uses  the  RCA  PRICE  Hardware  (H) 
parametric   cost   model   early   in   the  development   or 
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concept  exploration  stages  of  an  SOA  extension. 

The  PRICE  model  contains  thousands  of  mathematical 
equations  relating  the  input  variables  to  cost.  Each 
specific  set  of  input  parameters  uniquely  defines  the 
hardware  for  cost  modeling.  The  resultant  cost  output 
is  determined  from  the  mathematical  equations  alone. 
PRICE  does  not  perform  the  function  of  a  table  look- 
up model . 

PRICE  has  been  designed  to  estimate  costs  with  a 
minimal  amount  of  hardware  information.  This  feature 
makes  it  a  legitimate  tool  for  cost  estimation  of 
programs  in  concept  stage  of  development , since  the 
model  uses  its  internally  generated  values  for  any 
missing  input  variables  in  order  toestimate  cost.  Of 
course, it  is  always  preferable  for  the  PRICE  user  to 
supply  the  inputs,  when  their  values  are  known.  In 
this  way,  the  statistical  uncertainty  is  reduced. 
[REF.  6  :  p.  7] 

Lockheed  applies  the  PRICE  H  model  when  okey 
representative  system  characteristics  are  known. "J 

PRICE  H  also  identifies  cost  drivers  and 
quickly  and  economically  evaluates  SOA  extension  trade- 
off and  schedule  analysis. 

The  major  disadvantages  of  the  PRICE  H  model  as 
noted  by  LMSC ' s  senior  cost  estimating  managers  are: 

1.  Not  based  on  actual  history, 

2.  Algorithms  need  to  be  adjusted  for  different 
projects,  divisions  and  acquisition 
environments . 

PRICE  H  relies  on  input  of  a  subassembly  or 
higher  level  unit's  active  electronic  weight  and 
mechanical / structural  weight  as  the  most  critical  cost 
and  schedule  factors.  The  accuracy  of  the  PRICE  output 
is  directly  related  to  the  accuracy  of  the  weights 
input . 

Input   characteristics   of   the   PRICE   H  model 

which  enhances   its   value   as   an   SOA   extension  cost 

estimation  model  are: 

1.  Inputs  consider  the  lead  time  necessary  for 
set-up,  parts  procurement  and  redesign.  This  is 
significant  for  most  SOA  extensions. 


3  Information    obtained    from    LMSC    SOA   cost 
presentation,  September  24,  1987,  Sunnyvale,  CA . 
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2.  The  degree  of  new  design  and  complexity  of 
development  engineering  tasks  is  included. 

3.  The  operational  environment  in  which  the  system 
is  to  be  used  is  evaluated  in  cost  estimates. 
SOA  extensions  such  as  the  ANGAS  project  are 
designed  to  be  used  in  a  new  operational 
environment,  outside  the  earth's  atmosphere. 

4.  Type  and   manufacturing  complexity  of  the 
electronics  within   the  system   or  component  is 
evaluated  by   the  model.  New  electronics  design 
is  often  a  significant  factor  of   SOA  extension 
costs  . 

5.  The  model  also  evaluates  technological 
improvement,  year  of  technology  and 
technological  delay  or  lags.  These  inputs  are 
often  SOA  extension  cost  drivers. 

Illustration  2-1  is  a  sample  output   of  a  PRICE 

H   run   provided   by   LMSC   Cost   Modeling  and  Analysis 

Manager.  The   output   has   been   broken   down   into  six 

sections.  The  following  general  output  descriptions  are 

summarized  from  the  PRICE  Executive  Handbook   [REF.  7  : 

p.   20].   Relevant   SOA   extension   output  data  is  also 

highl ighted . 

Section  1:  identifies  time  of  the  run,  the  name  of 
the  item,  general  information  about  the 
item,  the  recurring  unit  production  cost 
and  monthly  production  rates.  Illus.  2-1 
identifies  the  item  as  a  pyro-kit,  to  be 
produced  in  a  quantity  of  one.   The  unit 
weight  was  input  at  7.96  pounds,  the  unit 
volume  was  .26  cubic  feet  and  mode  two 
designates  the  item  as  a  mechanical  item 
with  no  electronic  elements.   The  quantity 
/  next  higher  assembly  designator  is  a 
one  which  determines  the  number  of  units 
required  to  be  integrated  and  tested  at 
the  next  higher  assembly  level. 

Section  2:  lists  program  costs  in  engineering  and 

manufacturing  subtotals.  Illustration  2-1 
indicates  only  production  costs  of  1008 
dollars.  Development  costs  are  blank,  in- 
dicating that  no  development  costs  are 
required.  Many  SOA  extensions  require  sig- 
nificant development  costs.  The  ability  of 
PRICE  H  to  output   development  and 
production  costs  separately  is  useful  in 
analysis  of  technological  alternatives. 
Illus.  2-1  lists  engineering  costs  of 
$504.   These  costs  are  composed  of 
drafting  costs  of  $46.,  design  costs  of 
$250.,  project  management  costs  of  $148. 
and  data  costs  of  $60.   Manufacturing 
costs  total  $505.   These  costs  are 
composed  of  production  costs  of  $453.  and 
tools  and  test  equipment  costs  of  $52. 
Total  engineering  and  manufacturing  costs 
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equal  $1008. 

Section  3:  provides  output  useful  for  analysis  of  the 
inputs  with  known  standards.  Density. 
engineering  changes  and  mean  time  between 
failures  values  in  Illus.  2-1  are  followed 
by  an  asterisk.  This  indicates  that  these 
input  values  were  not  known  and  calculated 
by  the  model.  Many  inputs  required  to 
estimate  the  costs  of  an  SOA  extension  are 
not  available  during  the  early  phases  of 
development.  The  ability  of  the  PRICE  H 
model  to  derive  output  and  then  provide 
missing  input  is  ideal  for  SOA  extensions 
during  early  phases.   Illus.  2-1  shows 
mechanical  design  factors  that  were  input 
for  the  run.   The  manufacturing  complexity 
of  10.42  is  an  empiricaly  derived  value 
that  represents  the  product's 
producibil i ty  which  is  a  function  of 
material  type,  finished  density  and 
fabrication  methods.   The  mechanical 
integration  level  of  .7  represents  the 
level  of  mechanical  integration  and  test. 
This  factor  describes  the  level  of  effort 
required  for  the  integration  of  mechanical 
equipment  in  the  next  higher  assembly.  The 
platform  input  of  2.2  designates  the 
intended  operating  environment.   An  input 
of  2.0  or  more  represents  unmanned  or 
manned  space  operations.   The  year  of 
technology,  1985,  has  been  calculated  by 
the  program.   The  reliability  factor  of 
1.0  is  a  multiplier  used  to  deviate  the 
mean  time  between  failure  value  which  has 
been  calculated  by  the  model. 

Section  4:  outputs  development  and/or  production 

schedule  information.  Illus.  2-1  indicates 
that  the  dates  for  completion  of  the  first 
item  and  overall  completion  were  not  known 
before  the  run.  The  model  provides  valuable 
schedule  information  which  is  often 
difficult  to  determine  for  SOA  extensions. 

Section  5:  provides  additional,  in-depth  cost  output. 
The  model  again  provides  previously  unknown 
information  which  would  be  valuable 
for  SOA  extension  estimating.  First  unit 
production  costs  are  $452.17,  amortized 
unit  costs  are  $1008.30.  the  production 
tooling  cost  factor  is  i.O   and  the  unit 
learning  curve  is  86.4  percent. 
The  production  cost  multiplier  of  1.20  is  a 
multiplying  factor  used  to  include  mock-ups 
for  general  and  administrative  and  fee  or 
profit  in  the  production  cost  outputs. 

Section  6:  one  of  the  most  useful  outputs  of  the 
PRICE  H  model  is  the  cost  uncertainty 
measure.  Illus.  2-1  provides  a  cost  range 
of  954  to  1046  dollars.  SOA  extension 
development  and  production  costs  inherently 
contain  a  high  level  of  uncertainty 
due  to  the  technical  innovation  and 
engineering  complexity. 
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4 .     The  Lockheed  STAR  Cost  Estimating  Model 

Lockheed's  STAR  cost  estimation  model  combines 
the  characteristics  of  a  similar-to  model  and  an 
associated  model.  A  similar-to  methodology  uses 
historical  cost  data  from  tasks  or  equipment  similar  to 
the  equipment  under  evaluation.  A  high  degree  of 
accuracy  and  cost  breakdowns  to  the  subsystem  or  box 
level  (WBS  level  3  or  4)  is  provided. 

The  associated  program  methodology  uses  total 
annual  costs  from  technically  representative  projects 
that  have  been  adjusted  by  economic  and  complexity 
factors . 

The  basis  of  the  STAR  model  is  the  detailed 
historical  database  that  has  been  developed  over  a  20 
year  period.  This  data  base  represents  actual 
production  and  development  cost  experience,  rather  than 
success  oriented,  or  tailored  data. 

The  STAR  model  is  made  up  of  three  distinct 
estimating  models  with  separate  data  bases  for  each.  A 
flight  hardware  cost  estimator,  a  ground  hardware  cost 
estimator  and  a  software  cost  estimator.  The  individual 
models  use  an  algorithm  designed  for  its  particular 
line  of  business.  The  software  cost  estimator  works 
with  a  software  size  estimator  that  calculates  lines  of 
code  by  functional  application.  The  three  models  can  be 
used  in  an  integrated  manner  to  derive  a  total  system 
cost . 

Minimum  inputs  include:  program  hardware  / 
software  lists,  a  master  schedule  and  the  year  dollars 
required.  Critical  SOA  extension  estimating  factors 
such  as:  design  complexity,  escalation  of  costs,  and 
quantity  are  incorporated  in  the  STAR  model. 

The  primary  advantage  of  the  STAR  model 
compared  to  the  PRICE  H  model  is  that  output  is  derived 
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from  actual  or  historical  costs.  STAR  also  allows 
updating  of  costs  as  a  project  progresses  and  tradeoff 
and  "what  if  analysis". 

The  STAR  model  contains  a  program  loader  that 
pulls  the  required  type  of  data  from  the  data  base.  The 
FILGEN  pulls  analogous  cost  items  from  the  data  base. 
Each  item  on  the  program  hardware  list  is  coded  with  a 
box  status,  box  type  and  engineering  status. 

Box  status  codes  are  either:  make  (m),  buy  (b) 
or  government  furnished  equipment  (g). 

Box  type  codes  are  either:  development  (d), 
qualification  (q)  or  production  (p). 

Engineering  status  codes  are  either:  new  (n), 
modified  (m),  existing  (e)  or  production  (p). 

Each  item  on  the  program  hardware  list  is  also 
assigned  input  values  for:  quality,  management  effort, 
design  engineering  required,  systems  engineering 
required,  software  engineering  effort  required, the 
manufacturing  complexity,  test  requirements  and  product 
assurance . 

The  model  now  makes  a  cost  run,  the  Lockheed 
labor  rates  are  the  default  labor  rate  values.  Appendix 
C4  provides  sample  output  for  a  subsystem  program 
hardware  list. 

Final  output  includes:  total  hours,  dollar 
value  for  the  labor  hours,  material  costs  in  dollars 
and  total  dollar  value  for  each  item  on  the  subsystem 
hardware  list.  The  final  total  price  is  obtained  from 
analogous,  historical  cost  data.  If  any  of  seven  input 
values  were  not  initially  known  the  total  cost  is 
normalized  and  reallocated  back  to  the  missing  boxes. 


4  APPENDIX  C  is  not  part  of  the  unlimited 
distribution  of  this  thesis.  Individuals  within  DOD  may 
obtain  a  copy  of  APPENDIX  C  from  Professor  Willis  R. 
Greer,  Jr.,  Naval-  Postgraduate  School,  Monterey, 
Cal i  f ornia . 
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The  STAR  model  is  an  ideal  tool  for  Lockheed  to 
use  to  estimate  SOA  extension  costs.  The  data  base  is 
retrievable  by  line  item  or  box  level,  which  enables 
composite  data  from  even  classified  projects  to  be 
included  as  unclassified  useable  data. 

If  a  particular  line  item  on  the  hardware  parts 
requirement  list  is  going  to  be  subcontracted  or 
purchased  directly,  the  program  allows  these  known 
costs  to  be  entered.  Otherwise  a  data  base  is  selected 
which  matches  the  functional  and  characteristic 
qualities  of  the  item.  The  default  learning  curve 
value  is  95  percent. 

The  output  of  the  STAR  model  is  designed  to  aid 
SOA  cost  analysis.  Output  can  be  formatted  to  provide: 
development /non-recurring  costs  and  production  / 
recurring  costs.  Appendix  C  provides  sample  output  in 
this  format  and  an  example  of  output  in  WBS  reference 
number  sequence. 

The  accurate,  uniform  measurement  and 
correlation  of  historical  data  into  box  level  or 
component  factor  enables  extremely  precise  SOA 
extension  cost  estimations  by  the  STAR  model  once  parts 
requirements  are  well  defined. 
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ILLUSTRATION  2-1 


I  . 

INPUT  FILENAME:  EX    24  SEPT  87  16:56 
GLOBAL   FILENAME:  LMSCGL01 

PYRO   KIT 

PRODUCTION  QUANTITY 

UNIT  WEIGHT        7.96 

UNIT  VOLUME        0.26 

UNIT  PRODUCTION  COST 

MONTHLY  PRODUCTION  RATE 


1 

MODE 

2 

QUANTITY /NHA 

1 

452.67 

0.00 

II 


PROGRAM  COST 
($1000) 

ENGINEERING 
Drafting 
Design 
Systems 
Project  Mgmt 
Data 

Subtotal (ENG) 

MANUFACTURING 
Production 
Prototype 
Tool-Test  Eq 
Subtotal (MFG) 

TOTAL  COST 


DEVELOPMENT   PRODUCTION   TOTAL  COST 


46 
250 

46 
250 

148 

60 

504 

148 

60 

504 

453 

453 

52 
505 

52 
505 

1008 


1008 


III  . 

DESIGN  FACTORS        MECHANICAL 

Weight  7.960 

Density  30.038* 

Mfg.  Complexity  10.422 

Engineering  Change  0.033* 

Integration  Level  0.700 


PRODUCT  DESCRIPTORS 
Platform        2.200 
Year  of  Tech.    1985 
Reliability  Fac .  1.0 
MTBF(Field)    21971* 


IV. 

SCHEDULE  START 

Production    Jan  85 


FIRST  ITEM    FINISH 
(19)   Jul  86*  (0)   Jul  86*  (19) 
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ILLUSTRATION  2-1   (Continued) 


V. 

SUPPLEMENTAL  INFORMATION 

Economic  Base  185  Tooling  and  Process  Factors 

Escalation  0.00  Production  Tooling   1.00* 

T-l  Cost  452.17*Rate  Tooling            0 

Amor  Unit  Cost  1008 . 30*Pr ice  Info  Factor    0.950 

Prod  Cost  Mult  1.20  Unit  Learning  Curve  0.864 


VI  . 

COST  RANGES  DEVELOPMENT  PRODUCTION  TOTAL  COST 
From                  -            954  954 

Center  -  1008         1008 

To  -  1046  1046 
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Ill .  METHODOLOGY  AND  DATA 

A.  INTRODUCTION 

This  chapter  describes  the  methodology  of  the 
thesis  and  presents  data  that  was  collected  during  the 
study.  The  methodology  section  is  divided  into  two 
subsections,  data  collection  and  data  analysis. 

Data  includes:  estimated  costs  submitted  by 
Lockheed  in  the  ANGAS  contract  proposals;  Defense 
Contracts  Audit  Agency  (DCAA)  audits  of  the  ANGAS 
contract  proposals;  costs  questioned  in  the  DCAA 
audits;  final  negotiated  costs  with  justifications; 
actual  costs  incurred  at  different  stages  of  the 
contract;  statement  of  work  items;  a  listing  of  major 
mi lestones . 

B.  METHODOLOGY 

1 .   Data  Collection 

The  methodology  used  in  this  research  is  case 
study.  An  information  search  was  conducted  to  obtain 
background  data.  Automated  searches  were  conducted 
through  the  Naval  Postgraduate  School  library's  DOD, 
NON-DOD  and  Defense  Technical  Information  Center's 
files.  The  following  literature  provides  information 
on  SOA  extensions,  development  and  production  cost 
estimation  issues  and  a  background  that  will  enhance 
understanding  of  the  issues  in  this  case  study. 
Literature  that  is  pertinent  to  data  analysis 
methodology  will  be  presented  later  in  this  chapter. 

Captain  Helmut  W.F.  Scheel ,  USAF ,  in  his 
master's  thesis  [REF.  l:p.  1-91],  studied  the  methods, 
techniques  and  objectives  used  in  estimating  costs  for 
exploratory  development  projects  by  Air  Force  Research 
and   Development   Laboratories.     He  also  attempted  to 
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identify  factors  which  contribute  to  the  variance 
between  project  manager's  cost  estimates  and 
contractor's  proposal  costs.  Captain  H.  Scheel 
surveyed  45  government  project  managers  to  determine 
cost  estimating  techniques.  Important  conclusions  of 
his  study  include: 

1.  77.8  percent  of  the  program  managers  used 
historical  data  for  general  cost  estimating. 

2.  82.2  percent  of  the  program  managers  used 
historical  data  and  experience/ judgment  to 
estimate  labor  costs  and  manpower  requirements. 

3.  Low  cost  estimates  were  sometimes  made  because 
of  limited  funds  available  for  a  specific 
proj  ect . 

4.  The  major  variance  factor  attributable  to  the 
bidders  was  errors  in  estimation  as  a  result  of 
misinterpretation  of  the  Statement  of  Work 
(SOW). 

5.  Contractors  can  sometimes  propose  a  different 
level  of  effort  than  specified  in  an  SOA 
extension  due  to  technical  insight  and 
capabilities  known  only  by  the  contractor. 

6.  Establishment  of  a  computerized  system  with 
standardized  software  would  provide  a  data  bank 
that  would  enable  more  accurate  cost  estimation 
analysis  within  the  government. 

7.  Most  project  managers  do  not  accurately 
estimate  costs  for  new  projects.  The  majority 
underestimate  by  30  percent  or  less,  although 
some  underestimate  by  as  much  as  200  percent. 
[REF.  l:p.  33,  36,  41,  51,  67,  92-96] 

The   ANGAS   project,   like   most   SOA  extension 

efforts,  requires  a  long  research  and  development  (R&D) 

period.   To  accurately   estimate  and   evaluate  proposed 

contractor    costs    an    understanding   of   long-range 

corporate  R&D  planning  is  useful.   Ward  C.   Lowe  framed 

the  process  into  seven  significant  steps: 

1.  Specify  as  clearly  as  possible  the  basic 
technological  objectives  which  are  of  primary 
interest.  These  may  be  stated  broadly  at  the 
beginning  but  must  eventually  be  broken  down 
into  particular  areas. 

2.  Identify  the  goals  toward  which  the  company  is 
working,  which  ideally  are  set  forth  in  the 
corporate  and  the  research  objectives. 

3.  In  keeping  with  the  preceding  two  factors, 
seriously  and  imaginatively  consider  all 
possible  results  which  may  be  achieved  if  the 
research  efforts  are  successful. 
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4.  Rank  the  hypothetical  capabilities  of  the 
research  efforts  as  determined  in  the  preceding 
step,  in  terms  of  their  potential  contribution 
to  the  achievement  of  specific  goals.  The 
ranking  should  achieve  the  corporate  or  the 
research  objectives. 

5.  Outline  the  principle  technological  steps  which 
are  required  to  achieve  the  hypothetical 
research  results  listed  in  the  preceding  step. 
Such  an  outline  should  uncover  the  more 
significant  gaps  in  existing  knowledge. 

6.  Select  the  small  number  of  high-value  research 
results,  as  determined  previously,  which  have 
reasonably  well-specified  steps  and  a  minimum 
number  of  knowledge  gaps.  Give  these  results 
further  examination  and  consideration  for  full- 
scale  research  efforts. 

7.  Remain  alert  to  any  developments  that  could 
significantly  change  the  value  ratings  used  in 
the  preceding  steps.  When  research 
breakthroughs  fill  in  missing  knowledge  gaps, 
the  process  should  be  repeated.  [REF.  8:p.  62- 
63  J 

The  above  steps  are  also  useful  for  government 
program  managers  when  defining  the  technological  goals 
of  SOA  extension  projects  and  drafting  the  SOW. 
Accomplishment  of  SOW  items  during  the  development 
phase  of  an  SOA  extension  contract  may  be  the  primary 
measurement  tool  for  evaluating  technological  progress. 

The  bottom-up,  parametric  and  similar-to-cost 
estimating  models  described  in  Chapter  II  are  the  most 
widely  used  cost  estimating  methods.  References  4,  6, 
and  7  provide  information  for  further  study. 

It  was  time  consuming  and  difficult  to  identify 
a  company  willing  to  cooperate  in  a  case  study  which 
offered  an  SOA  extension  project  useful  for  analysis. 
Several  Marketing  and  Research  and  Development 
Directors  were  contacted  by  telephone  to  determine 
interest  in  and  suitability  for  study. 

Companies  initially  contacted  included  GTE 
Inc.;  National  Semi-Conductor  Inc.;  Fairchild  Inc.; 
Advanced  Microcircuit  Devices  Inc.;  Cypress  Electronics 
Inc.;   Acrian    Inc.;'  Hewlett    Packard   Inc.;   Harris 
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Microwave   Inc ;   Intel   Corp.   Inc.;   Litton  Industries 
Inc.;  and  Eaton  Semiconductors  Inc. 

Company  responses  ranged  from  "not  interested" 
to  enthusiastic  requests  for  additional  information 
about  the  research  effort.  Companies  expressing  an 
interest  in  aiding  the  research  effort  were  then 
evaluated  on  their  ability  to  provide  viable  projects 
for  study.  Three  other  Naval  Postgraduate  School 
students  had  selected  SOA  extension  case  studies  for 
their  theses.  All  of  us  worked  as  a  team  to  identify 
appropriate  firms.  These  efforts  resulted  in  the 
selection  of  GTE  Inc.  and  Litton  Industries,  Inc.,  as 
candidates  for  study  by  two  other  graduate  students. 

LCDR.  Fred  Voellm,  SC ,  USN ,  of  the  Navy 
Procurement  Office  located  at  LMSC ,  was  contacted  for 
assistance  in  identifying  a  government  contract 
administered  by  his  office  for  an  SOA  extension  that 
could  be  studied.  LCDR.  Voellm  produced  a  list  of 
contracts  that  were  considered  high  technology  and 
potential  candidates  for  study. 

Individual  contracts  were  evaluated  on  the 
degree  of  new  technologies  and  SOA  extension,  the 
availability  of  initial  and  actual  cost  data  and  the 
stage  of  completion  of  the  contract. 

The  ANGAS  experiment  program  initiated  by  the 
Defense  Advanced  Research  Projects  Office  in 
conjunction  with  the  Research  and  Development  Division 
of  LMSC  was  selected  because  it  combined  numerous  SOA 
technological  extensions  into  a  design  that  was  unique 
and  required  operation  in  space.  Selection  of  the 
ANGAS  project,  which  is  still  in  early  research  and 
design  stages,  provided  an  opportunity  to  study 
development  costs  and  how  technological  progress  is 
evaluated . 


31 


Mr.  Don  Dorsett,  LMSC ' s  ANGAS  contract 
administrator,  was  visited  numerous  times  to  obtain 
background  and  cost  data.  The  ANGAS  "Weekly  Financial 
Status  Report"  and  "Work  in  Process  Ledger  by  Work 
Order"  provided  actual  current  costs  and  spending 
rates.  A  summary  of  this  data  will  be  presented  in  the 
data  section  of  this  chapter.  Mr.  Dorsett  also 
discussed  the  significant  technological  and  SOA 
extension  characteristics  of  the  ANGAS  project.  These 
are  listed  in  Chapter  II. 

LTCOL.  George  P.  Lasche,  USA,  Geophysical 
Sciences  Division,  Defense  Advanced  Research  Projects 
Office,  Arlington,  VA ,  is  the  government's  program 
manager  for  the  ANGAS  program.  LTCOL  Lasche,  USA,  was 
interviewed  by  telephone.  The  project's  technological 
objectives,  goals,  potential  applications,  program 
status  and  evaluation  techniques  used  to  analyze 
proposed  contractor  costs  will  be  presented  in  the  data 
section  of  this  chapter. 

The  government  program  manager  for  the  ANGAS 
program  authorized  access  to  the  principle  investigator 
at  the  Lockheed  Space  Sciences  Laboratory  in  Palo  Alto, 
CA ,  Dr.  George  Nakano .  Lockheed's  principle 
investigator  for  the  ANGAS  project  is  responsible  for 
technical  supervision  and  overall  project  direction. 
Dr.  Nakano  was  interviewed  by  telephone  and  visited  at 
the  Space  Sciences  Laboratory.  His  description  of  the 
ANGAS  cost  estimation  effort,  accomplishment  of 
technological  goals,  spending  status,  problem  areas  and 
significant  SOA  extension  achievements  will  be 
presented  in  the  data  section  of  this  chapter,  Appendix 
B  and  Chapter  IV. 

Cost  estimation  procedures  used  by  Lockheed 
were  obtained  from  Lockheed's  Estimating  Systems 
Description   Manual   and   from   a   presentation   on  SOA 
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pricing  by  Mr.  Ted  Castro,  Manager  of  Lockheed's 
Estimating  Systems;  Mr.  Donald  H.  Palmby,  Manager  of 
Lockheed's  Cost  Modeling  and  Analysis;  and  Mr.  Ken 
Peeler,  Chief  Industrial  Accountant  of  Lockheed's 
Astronautics  Division. 

Information  and  characteristics  of  bottom-up 
cost  estimating,  parametric  cost  models  and  Lockheed's 
internally  developed  STAR  cost  estimation  were 
obtained.  Input  parameters  and  output  data  were 
obtained  for  sample  SOA  extension  component  cost 
estimates  using  the  parametric  and  STAR  cost  estimation 
models.  This  information  was  detailed  in  Chapter  II. 
2 .   Data  Analysis 

The  estimated  ANGAS  development  costs  as 
provided  in  Lockheed's  contract  proposal  and  modified 
after  government  audit  and  negotiation  are  compared  to 
actual  reported  costs.  Differences  and  trends  in 
specific  cost  categories  are  analyzed.  Overall 
spending  rates  are  compared  with  anticipated  spending. 
The  results  of  the  data  analysis  are  presented  in 
Chapter  IV. 

Significant  characteristics  of  Lockheed's  cost 
estimating  methods  and  models  that  are  potentially 
useful  in  aiding  better  estimation  of  SOA  extension 
projects  by  the  government  are  also  recorded  in  Chapter 
IV. 

The  specific  methodology  that  will  be  used  in 
the  cost  analysis  will  closely  follow  Dearden's  simple 
variance  analysis  as  detailed  in  Cost  and  Budget 
Analysis .  Actual  costs  will  be  obtained  from  the  ANGAS 
"Weekly  Financial  Status  Report"  and  compared  with  the 
estimated  or  "budgeted  costs"  in  a  variance  matrix 
format . 

Budgeted  costs  will  define  the  planned 
expenditure   for   work   scheduled    for   accomplishment 


33 


during  a   specific  phase  of  the  contract.   The  budgeted 

cost  from  work  scheduled  (BCVS)  for   a  level   of  effort 

contract,   like    the   ANGAS   contract,   "is   based   on 

expected  expenditures  by  tasks  to  be   completed."  [REF. 

9:p.  6] 

A  determination  of  the  degree  of  completion  for 

stated   technological   goals   and   task   completion   is 

provided  in   Chapter  IV.   Information  obtained  from  the 

government's  program  manager   and   the   ANGAS  principle 

investigator  about   the  status  of  the  SOW  items  will  be 

the   basis   for   establishing   task   and   technological 

progress.   The  reasons  why  the  program  manager  was  able 

to  : 

specify  as  clearly  as  possible  the  basic 
technological  objectives  which  are  of  primary 
interest  [REF.  8 : p .  62] 

will  also  be  discussed  in  Chapter  IV. 

C .   DATA 

1 .   Introduction 

The  Research  and  Development   Division  of  LMSC , 

Palo  Alto,   CA ,  has   been  funded  through  an  incremental 

cost-plus-fixed-fee  contract  to  develop  the  ANGAS.   The 

major  milestones  are  presented  in  Appendix  B. 

In  a  cost-plus-fixed-fee  contract,  the  government 
pays  allowable  cost  and  a  fixed  fee.  Fixed  fee  does 
not  vary  with  actual  costs.  Provides  minimum 
incentive  for  contractors  to  control  costs. 
Completion  requires  contractor  to  deliver  end 
product.  Requires  specified  level  of  effort  over 
stated  period  of  time.  Used  most  widely  for 
research.  preliminary  exploration  or  study.  [REF. 
10 : p .  49 ] 

The   following   information   details   ANGAS   funding  to 

date  : 

PERIOD EST.  COST FIXED  FEE     COST  PLUS  FEE 

1  $  1,281,655       $   68,345     $  1,350,000 

2  $  3,392,620      $  192,843     $  3,585,463 

TOTALS     $  4,674,275-     $  261,188     $  4,935,463 
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LMSC  submitted  a  contract  proposal  on  November 
26,  1985,  after  the  first  incremental  funding  had  been 
approved,  that  covered  a  78  month  period  commencing 
April  1,  1986  through  completion.  The  government 
choose  not  to  allocate  funds  for  this  entire  6  1/2  year 
period  and  has  incrementally  funded  the  project.  The 
contractor's  proposed  cost  for  completion  of  ANGAS  was 
516,589,398.00  ( CPFF )  .  Appendix  B  contains  the  SOW 
items  from  the  two  incrementally  funded  contracts  and 
the  unfunded  proposed  completion  contract. 

D.   ESTIMATED  COSTS  FOR  FIRST  CONTRACT  PHASE 

LMSC,  Research  and  Development  Division  submitted 
the  following  breakdown  of  proposed  costs  for  the 
initial  ANGAS  contract: 

PRICE  BREAKDOWN 
DESCRIPTION  TOTALS 

LABOR  HOURS  14  ,688 

LABOR  DOLLARS  $   319,630 

OVERHEAD  506,192 

G  &  A  EXPENSES  134,942 

MATERIALS  110,793 

SUBCONTRACT  146,700 

PROCUREMENT  BURDEN  9,012 

TRAVEL  16,064 

SUB.  COSTS 
COST  OF  MONEY 

TOTAL  COST 

FIXED 

TOTAL  CPFF 

The  individual  cost  element  totals  were 

developed  as  follows: 

1.  Direct  Labor  -  direct  labor  hours  are  based  on 
a  technical  evaluation  developed  from  detailed 
analysis  of  the  program  requirements.  The 
labor  rates  are  current  averages  for  direct 
labor  pools,  by  skill  categories. 


35 


$1 

,243 

333 

38 

,322 

$1 

,281 

,655 

99 

,647 

$1 

,381 

,  122 

HOURS 


RATE 


AMOUNT 


DESIGN  ENGINEER 

DESIGN  ENGINEER    SENIOR 

DESIGN  SPECIALIST 

ENGINEER/SCIENTIFIC  CLERICAL  AID 

LABORATORY  ANALYST  -  RESEARCH 

RESEARCH  ENGINEER 

SCIENTIST,  SENIOR 

SCIENTIST  -  RESEARCH 

STAFF  ENGINEER 

STAFF  SCIENTIST 

TECHNICAL  PUBLICATIONS 

TOTAL  DIRECT  LABOR 

2.     Overhead  rates   are  applied   as  a  percent  times 
direct  labor  dollars.   The  rates   used  for  1985 
and  1986  were  1.592  and  1.560,  respectively. 
These  rates  apply  to  R  &  D  projects. 


2 

,324       S 

518 

.31    $42 

,5  46 

2 

,656       J 

522 

.08    S58 

,6  38 

664       J 

526 

.13    S17 

,350 

179       SI 2 

.46    8    2 

,230 

1 

,162       816 

.67    819 

,376 

1 

,148       818 

.87    821 

,661 

1 

,328       817 

.89    823 

,755 

2 

,656       824 

.24    864 

,391 

621       S28 

.07    S17 

,430 

1 

,771       S28 

.01    S49 

,608 

179     r 

,  688    $c 

514 

.78    $    2 

,645 

14 

21.76    $319 

,630 

OVERHEAD 


TOTAL  OVERHEAD 


1985 
1986 


BASE 

$236,523 
$83, 107 

$319,630 


RATE 

..592 
..560 


AMOUNT 


;376 

;i29 


545 
647 


$506  ,192 


3. 


General 
appl ied 
dol lars . 
.424  and 


and   Administrative   Expenses  are  also 
as    a   percent    times   direct   labor 
The  rate  used  for  1985  and  1986  were 
.417,  respectively. 


G  &  A  EXPENSES 


TOTAL  G  &  A  EXPENSES 


1985 
1986 


;236 
i  83 


BASE 

523 

107 


RATE 

1.424 
1.417 


AMOUNT 


100 

34 


286 
656 


$319 ,630 


$134  ,942 


Material  requirements  were  listed,  identified 
and  priced  by  component.  Costs  of  purchased 
services  are  also  included  in  this  cost 
element . 


MATERIAL 
PURCHASED  SERV. 

TOTAL  MATERIALS 


1985 
1985 


$51  ,547 
$59,246 

$110  ,793 


Subcontract  cost  estimates  are  based  on  actual 
price  quotes  received  from  the  subcontractors 
in  response  to  a  RFP .  Subcontract  costs  for 
1985  were  estimated  at  $146,700. 

Procurement  burden  is  applied  at  the  rate  of 
.035  of  the  total  estimated  costs  for 
materials,  purchased  services  and  subcontract 
costs.  The  rate  is  developed  from  historical 
data. 


1985 
TOTAL  PROCUREMENT  BURDEN 
7  . 


BASE 
257  ,493 


RATE   AMOUNT 

0.035   $9  ,012 

$9  ,012 


Travel  -  per  diem  rates  are  based  on  historical 
data  and   air  fare   costs  are   based  on  tourist 
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class   rates.      Travel 
project  requirements. 


AIR  FARE 
ALBUQUERQUE 
2  TRIPS  OF  2 
TECH  MEETING 

DENVER 

2  TRIPS  OF  2 

INTEGRATION 


reflects   anticipated 
OTY   RATE   AMOUNT 


MEN  FOR  3  DAYS 


MEN  FOR  3  DAYS 
COORDINATION 


LOS  ANGELES/BURBANK 
3  TRIPS  OF  2  MEN  FOR 
FLT.  COORD.  MEETING 


1  DAY 


1985 


1985 


1985 


$  193 


772 


$  325   1,300 


$  145 


870 


WASHINGTON/ BALTIMORE 
6  TRIPS  OF  2  MEN  FOR  3 
VENDOR  &  CUST  COORD. 

PER  DIEM 
ONE  DAY  TRIPS 
WEST  COAST  TRIPS 
EAST  COAST  TRIPS 


DAYS 


1985   12   $  556   6,672 


6    $   45  270 

24    S   91  2,184 

36    $  111  3,966 

TOTAL  TRAVEL    $  16,064 


8.  Cost  of  money  is  an  element  of  facilities 
capital.  Rates  are  applied  to  direct  labor 
dollars,  general  and  overhead  base  and  the 
procurement  burden  base. 


DEVELOPMENT-LMSC 
G  &  A  EXPENSES 
PROCUREMENT  BURD 

TOTAL  COST  OF  MONEY 


1985 
1986 
1985 
1986 
1985 


S 

s 

s 
$ 


236 
83 

236 
83 

257 


BASE 

523 
107 
523 
107 
493 


RATE 

10310 
12283 
01055 
01254 
00074 


AMOUNT 


S 

S 


24 

10 

2 

1 


386 
208 
495 
042 
191 


$  38,322 


The  total  of  the  cost  elements  is  $1,281,655. 
Lockheed  submitted  the  contract  proposal  with  a  fixed 
fee  of  $99,467,  8  percent  of  costs.  The  total  CPFF 
requested  by  Lockheed  for  the  first  phase  of  the 
contract  was  $1,381,122. 

E.   DCAA  AUDIT  REVIEW 

Lockheed's  cost  proposal  was  audited  by  DCAA, 
ounnyvale,  CA .  The  results  summarized  below  are  set 
forth  in  audit  report  7481  -  5G  210091,  dated  5  June 
1985. 
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COST  CONTRACTORS      QUESTIONED 

ELEMENT  PROPOSAL  COSTS 

SUBCONTRACTS  $  146,700       $   25,379 

PURCHASED  SERVICES  59,246           59  246 

MATERIAL  51,547           51,547 

PROCUREMENT  BURDEN  9,012 4  ,766 

$  266,505      $  140,938 

DCAA   questioned   the   subcontract   costs  of   $ 

146,700  because  a  cost  or  price   analysis  had   not  been 

completed   by    LMSC .     Cost   or   price   analysis   for 

subcontracts   is   required   by    FAR   15.806.      DCAA, 

therefore,  applied  a  17.3  percent  negotiation  reduction 

factor  to  the  proposed  costs.   The  17.3   percent  factor 

is  based   on  DCAA   evaluation  of  subcontract  history  in 

the  Astronautics  and  Space  Systems  Division. 

The  purchased  services  of  $  59,246  and  material 

of  $51,547  were  questioned  because  LMSC  did  not  provide 

any  data  in  support  of  the  proposed  costs. 

The  procurement  burden  was   reduced  due   to  the 

above   questioned   costs   and   the   reduced  procurement 

burden  base. 

QUESTIONED  BASE  COSTS   $136,172 
PROPOSED  RATE  X   3.5$ 


TOTAL  QUESTIONED    $   4,766 

1 .   Summary  Of  Cost  And  Fee  Negotiations 

LMSC  submitted  an  SF  1411  with  supporting 
documentation  to  justify  the  questioned  subcontractor 
costs.  This  documentation  was  reviewed  and  approved  by 
the  negotiator  and  scientific  officer. 

LMSC  also  provided  a  detailed  breakdown  of  the 
questioned  costs  for  purchased  services  and  materials 
which  was  approved  by  the  government.  These  price 
breakdowns  were  based  on  past  experience  on  similar 
projects,  such  as  project  Winkler,  catalog  prices  and 
vendor  quotes . 

All  questioned  costs  were  reviewed  and  accepted 
after   negotiation.     LMSC   had   requested   a  fee  of  8 
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percent  of  costs  ($99,467),  the  government's  fee 
objective  was  4.2  percent.  As  a  result  of 
negotiations,  the  settled  fee  was  5.5  percent 
($68,345).  The  final  estimated  costs  for  the  first 
phase  of  the  ANGAS  project  was: 
ESTIMATED  COST       FIXED  FEE  TCFF 


SI ,281 ,000 


$   68,345 


$1  ,350 ,000 


F.   ESTIMATED  COSTS  FOR  SECOND  CONTRACT  PHASE 

LMSC   submitted   a   cost  proposal  dated  26  November 

1985  in  the  amount  of  $16,589,398,  covering  a  period  of 

performance  of   six  and   one-half  years.    Audit  report 

7481   -   6G210136   conducted   by   DCAA ,   Sunnyvale,  CA , 

questioned  portions   of  every  item  in  the  budget.   LMSC 

was  unable   to  substantiate   its  cost   estimates  to  the 

satisfaction   of   the   government.    LMSC  was  unable  to 

accurately  project  costs  six  years  into   the  future  for 

a  program  still  in  the  design  phase. 

It  was   decided  to   contract  only  for  the  final 

design   and   sensor   shield   evaluation   of   the   ANGAS 

project.    The  SOW   is  illustrated  in  Appendix  B.   LMSC 

proposed  a  cost  of  $  3,694,580.   The   costs  were  broken 

down  into   the  same  cost  elements  as  the  first  contract 

detailed  earlier  in  this  chapter. 

1 .   DCAA  Audit  Review  And  Negotiation 

The  following   proposed   costs   were  questioned 

and  negotiated: 

1.  Labor  Rates  -  LMSC  used  an  escalation  factor  of 
4.5  percent.  Due  to  the  government  ceiling  on 
salary  escalation,  LMSC  was  instructed  to 
recompute   labor   rates   using   a   3.5   percent 


escalation 


CATEGORY 


TECHNICAL  SPECIALIST 
TECHNICAL  STAFF 
TECHNICAL  SENIOR 
TECHNICAL  ENGINEER 
TECHNICAL  SUPPORT 
LABORATORY  SERVICES 
TECHNICAL  PUBS. 


PROPOSED 
RATE 


$  24 .66 
Vt  30.64 
S  21.45 
S  17.91 
S  13.60 
8  17.27 
$  15. 94 


NEGOTIATED 
RATE 

$  24 .42 
S  30.35 
S  21.24 
S  17.74 
S  13.47 
8  17.11 
$  15.78 


TOTAL 
HOURS 

10  ,210 

6  ,512 

10  ,460 

9  ,559 

812 

2  ,006 

493 
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2.  Fee  Negotiation  -  LMSC  requested  a  fee  of  9 
percent  of  costs,  exclusive  of  cost-of -money . 
The  government's  fee  objective  was  4.9  percent. 
A  fee  of  5.8  percent,  exclusive  of  cost-of- 
money ,  was  negotiated. 

2 .   Summary  Of  Estimated  Costs  For 
The  Second  Contract  Award 

The  following  chart  provides  the  final 
estimated  costs  for  the  second  contract  award  after 
audit  and  negotiation: 

PRICE  BREAKDOWN 
DESCRIPTION  TOTAL 

Labor  Hours 
Labor  Dollars 
Overhead 
G  &  A 
Material 

Purchased  Services 
Subcontracts 
Procurement  Burden 
Travel 

Subtotal 
Cost  of  Money 

Total  Cost 

FEE 

CPFF  $3,585,463 

G.   ACTUAL  COST  DATA 

The  level  of  funding,  actual  expenditure  level  and 
planned  spending  as  of  June  30,  1987  is  illustrated 
graphically  on  the  following  page: 
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052 

$   891 

,783 

1  ,302 

,025 

361 

,712 

583 

,236 

6 

,000 

102 

,968 

22 

,150 

31 

,657 

$3 

,301 

,531 

91 

,089 

$3 

,392 

,620 

192 

,843 
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COST         STATUS 


a 

0 

H 

n 


3  3 
0 


8  - 


7  - 


5  - 


5  - 


3  - 


Ai    or    June    30.    1967 


Month    mJ(ji|    on    mirk 


1987   — 


1988 


Actual  cost  data  were  obtained  from  Mr.  Don 
Dorsett,  LMSC ' s  contract  administrator  for  the  ANGAS 
project.  The  source  of  the  following  spending  data  is 
the  Weekly  Project  Financial  Status  Report,  dated 
September  31,  1986,  which  reflects  spending  at  the  end 
of  the  first  contract  phase. 

ACTUAL  COSTS 


FIRST 

CONTRACT  PHASE 

EXPENDED 

TO  DATE 

BUDGET 

LABOR  HOURS 

17,424  hours 

13  ,  900 

LABOR  COSTS 

$ 

369  ,323 

$ 

302  ,464 

OVERHEAD 

$ 

556  ,858 

$ 

510  ,392 

G  &  A  EXPENSES 

$ 

184  ,323 

$ 

147 ,780 

OVERTIME  PREMIUM 

$ 

24 

$ 

0 

MATERIAL 

$ 

15 ,533 

$ 

110  ,793 

PURCHASED  SERVICES 

$ 

36  ,099 

$ 

0 

SUBCONTRACT 

$ 

0 

$ 

146  ,700 

PROCUREMENT  BURDEN 

$ 

1  ,678 

$ 

9  ,  012 

TRAVEL 

$ 

6  ,657 

$ 

16  ,  064 

GRAND  TOTAL 


$  1  ,  170  ,  515   $  1  ,243 ,205 
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The  difference   between  the  amount  funded  under 

the  contract  and  the  budgeted  total  shown  above   is  the 

cost  of  money  and  fixed  fee  amounts. 

TOTAL  FUNDED  ($1,350,000)  =  TOTAL  BUDGETED  ($1,243,205) 
+  COST  OF  MONEY  ($38,450)  +  FIXED  FEE  ($68,345). 

Chapter  IV  discusses  the  actual  "budgeted" 
accomplishment  and  measure  of  technological  progress 
for  the  first  and  second  phases  of  the  contract. 

The  following  cumulative  actual  costs  as  of 
September  31,  1987,  provide  actual  expenditures  for  the 
first  and  second  contract  phases. 

ACTUAL  COST 


FIRST  AND  SECOND 

PHASES 

EXPENDED 

TO  DATE 

BUDGET 

LABOR  HOURS 

55, 

585  hours 

73,653 

LABOR  COSTS 

$ 

1 

,170,014 

$ 

1 ,369,867 

OVERHEAD 

$ 

1 

,806,416 

$ 

2 ,023,831 

G  &  A  EXPENSES 

$ 

542,392 

$ 

705  ,570 

OVERTIME  PREMIUM 

$ 

118 

$ 

100 

MATERIAL 

$ 

360,671 

$ 

196  ,203 

PURCHASED  SERVICES 

$ 

59,110 

$ 

61  ,100 

SUBCONTRACT 

$ 

0 

$ 

146  ,700 

PROCUREMENT  BURDEN 

$ 

13,116 

$ 

12 ,380 

TRAVEL 

$ 

25  ,606 

$ 

18,064 

OTHER 

$ 

325 

$ 

125 

GRAND  TOTAL  $  3,977,768   $  4,533,950 

The  difference   between  the  amount  funded  under 

the  first  and  second   contract  phases   and  the  budgeted 

total  shown   above  is   the  cost   of  money  and  fixed  fee 

amounts . 

TOTAL  FUNDED  ($4,935,463)  =  TOTAL  BUDGETED  ($4,533,950) 
+  COST  OF  MONEY  ($140,325)  +  FIXED  FEE  ($261,188). 


H.   INFORMATION  FROM  THE  GOVERNMENT'S  PROGRAM  MANAGER 

The   following   discussion   is  based  on  a  telephone 

conversation   with   LTCOL   G.   Lasche,   USA,  the  ANGAS 

program  manager   on  September   3,  1987.   LMSC  initially 
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proposed  the  ANGAS  project  to  the  government.  At  the 
time,  Lockheed  had  recently  completed  the  four  million 
dollar  Winkler  project.  Winkler  was  a  scaled  down 
aircraft-borne  spectrometer  that  utilized  many  of  the 
advanced  technologies  that  were  to  be  used  in  the  ANGAS 
proj  ect . 

Winkler  was  successfully  developed,  tested  and 
flown  at  the  estimated  costs.  LMSC ' s  proposed  costs 
for  ANGAS  were  evaluated  using  Winkler's  historical 
cost  data. 

ANGAS'  technology  was  more  advanced  than 
Winkler's  because  it  is  designed  to  function  in  space 
rather  than  the  earth's  atmosphere.  According  to  the 
program  manager,  projects  like  ANGAS  that  incrementally 
advance  the  SOA  are  usually  completed  much  closer  to 
the  scheduled  time  and  budgeted  costs  than  projects 
that  attempt  quantum  leaps. 

The  incremental  funding  of  the  ANGAS  contract 
allows  a  structured  development  and  technological 
evaluation  at  key  development  steps.  LTCOL.  Lasche, 
USA,  stated  the  contract's  SOW  items  and  required  tests 
stipulated  in  each  contract  phase  provide  the  primary 
evaluation  criteria  to  measure  technological  progress. 
To  date  all  SOW  items  and  tests  have  been 
satisfactorily  completed  on  time. 

It  was  also  noted  that  labor  costs  are  usually 
the  most  significant  expense  during  the  research  and 
development  phases  of  an  SOA  extension  project. 
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IV.  DATA  ANALYSIS 

A.  INTRODUCTION 

This  Chapter  analyzes  the  data  presented  in  Chapter 
III.  The  analysis  examines  cost  variances,  schedule 
variances  and  the  measurement  of  technological 
progress.  Significant  characteristics  of  Lockheed's 
cost  estimating  methods  and  models  that  are  potentially 
useful  in  aiding  better  estimation  of  SOA  extensions  by 
the  government  will  also  be  discussed.  The 
contributions  and  role  of  the  government's  program 
manager  for  the  ANGAS  will  also  be  highlighted  when 
appl icable . 

B.  COST  VARIANCE  ANALYSIS 

Table  1  provides  the  dollar  value  and  the 
percentage  of  cost  variances  by  cost  element  for  the 
first  funding  period  of  the  ANGAS  project  as  of  31 
September  1986: 


COST      ACTUAL 
ELEMENT   COSTS 

LABOR     $  369,323 
OVERHEAD    556,858 
G  &  A       184 ,323 

OVERTIME         24 
MATERIAL     15,533 
PUR.  SERV.   36,099 
SUBCONTRACT       0 
PRO.BURD.     1,678 
TRAVEL        6,657 

TABLE  1 

COST  VARIANCES 

BUDGETED   COST 
COSTS      VARIANCE 

$  302,464   $  -66,859 

510,392     -46,466 

147,780     -36,543 

0-24 

110,793     +95,260 

0     +36,099 

146,700    +146,700 

9  ,012     +  7 ,334 

16,064     +  9,407 

VARIANCE  AS 
%    OF  BUDGET 

22.10  % 

9.  10 

24.73 

NA 

85.98 

NA 

100 

81.38 

58.55 

TOTAL    $1,170,515 

$1 

,243,205 

+$72,690 

5.8£ 
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Table  1  indicates  that  negative  cost  variances5 
during  the  first  funding  phase  of  the  ANGAS  contract 
were  realized  in  labor  costs,  overhead,  G  &  A  expenses 
and  overtime  premiums.  These  cost  elements  are 
influenced  by  labor  costs  either  directly  or  through  an 
applied  rate  to  a  direct  labor  base. 

LMSC  had  budgeted  for  13,900  labor  hours.  The 
number  of  labor  hours  actually  expended  during  the 
first  phase  of  the  contract  was  17,424  hours.  Dr. 
George  Nakano ,  LMSC ' s  Principle  Investigator  for  the 
ANGAS  program,  explained  that  during  the  early  stages 
of  development  additional  research  and  development 
effort  was  put  into  the  sol id-cryogen  cooler  system. 
This  was  a  critical  item  in  the  overall  SOA  extension 
effort  that  has  a  long  lead-time  and  would  be  built 
outside  the  Space  Sciences  Laboratory  Division. 

All  other  cost  elements  experience  positive  cost 
variances  during  the  initial  funding.  A  cumulative 
positive  cost  variance  of  $72,690,  5.8  %  of  budgeted 
total  costs  was  reported  at  the  conclusion  of  the  first 
contract  period. 

Budgeted  labor  costs,  overhead,  G  &  A  expenses  and 
overtime  premiums,  all  based  on  direct  labor  costs, 
were  budgeted  at  77.27  %  of  total  budget  costs.  Actual 
costs  for  these  cost  elements  accounted  for  94.87  %  of 
actual  total  costs. 

Table  2  provides  the  cumulative  and  percentage 
variances  for  the  first  and  second  contract  phases  as 
of  31  September  1987: 


4   Negative   cost   variances   result   when  actual  costs 
exceed  budgeted  costs. 
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TABLE  2 

CUMULATIVE  COST  VARIANCES 

COST         ACTUAL   BUDGETED  COST       VARIANCE  AS 
ELEMENT      COSTS    COSTS     VARIANCE  %    OF  BUDGET 

LABOR     $1,170,014  $1,369,867  +$199,853  14. 59$ 

OVERHEAD   1,806,416  $2,023,831  +217,415  10.75 

G  &  A        542,392     705,570  +163,178  23.13 

OVERTIME          118         100  -      18  18 

MATERIAL     360,671     196,203  -164,468  83.82 

PUR.SER.       59,110      61,110  +   2,000  3.2 

SUBCONT.            0     146,700  +146,700  100 

PRO.BURD.     13,116      12,380  -     736  5.9 

TRAVEL        25,606      18,064  -   7,542  41.75 

OTHER 32_5 125  -     200 160 

TOTAL    $3,977,768   $4,533,950   +566,182     12.26^ 

Table  2  indicates  that  negative  cost  variances 
existed  at  the  end  of  the  second  funding  phase  in 
materials,  procurement  burden,  travel  and  other,  cost 
element  catagories.  A  cumulative  positive  cost 
variance  of  $556,182,  12.263  percent  of  budgeted  total 
costs,  was  reported  at  the  conclusion  of  the  first  two 
contract  periods. 

LMSC  had  budgeted  for  73,653  labor  hours  at  this 
state  of  the  development  effort.  The  cumulative  number 
of  labor  hours  actually  expended  during  the  first  two 
contract  phases  was  55,585  hours. 

Direct  labor  and  direct  labor-based  costs  were 
budgeted  at  90.4  %  of  total  budgeted  costs.  Actual 
labor-related  costs   were  88.4  %    of  actual  total  costs. 

Total  actual  costs  for  the  ANGAS  project  have  been 
slightly  less  than  budgeted.  Dr.  Nakano  and  Mr. 
Dorsett,  of  LMSC,  explained  that  the  slightly  slower 
overall  spending  rate  was  partially  attributed  to  the 
uncertainties  and  delays  experienced  in  obtaining 
timely  approval  of  incremental  funding  requests. 
Material  procurement  and  research  efforts  that  are 
needed  for  SOW  items  defined  in  future  phases  of  the 
project  would  have  been  initiated  in  the  first  and 
second  phases  of  the  project  if  future  funding  had  been 
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approved.  The  incremental  funding  approach  adopted  by 
the  government  provides  control  over  the  project  but 
also  makes  the  development  more  difficult  for  the 
contractor . 

Incentives  and  definite  evaluation  periods  for  a 
level  of  effort  contract  like  the  ANGAS  project  are 
established  through  the  incremental  funding, 
establishment  of  SOW  goals  for  each  contract  phase  and 
the  ability  to  adjust  future  funding  based  on  an 
evaluation  of  past  performance  and  future  potential. 

The  contractor  has  been  provided  incentive  to 
satisfactorily  achieve  stated  SOW  goals  for  the  phase 
of  the  contract  for  which  funding  has  been  reviewed. 

Future  long  lead-time  items  required  in  later 
contract  phases  of  the  ANGAS  project  cannot  be 
initiated  until  funding  of  those  contract  phases  is 
approved . 

C.   SCHEDULE  VARIANCE  AND  TECHNOLOGICAL  PROGRESS 

Schedule   variances   provide   a   way   to   highlight 

situations  where  effort  is  behind  or  ahead  of  schedule. 

The   traditional   methodology   for   analyzing   schedule 

variances  compares  the  budgeted  cost  of   work  performed 

and  the  budgeted  cost  of  work  scheduled.   In  a  level  of 

effort  development  contract,  such  as  the  ANGAS  project, 

a  determination   of  the  budgeted  cost  of  work  performed 

becomes   almost   impossible   to   calculate   due   to  the 

intangible   nature   of   the   goals   and   SOW  items.   An 

evaluation   of   technical   progress   is   a   more  useful 

measure  when   used  with  the  project's  cost  variances  to 

evaluate  overall  benefits  and  project  status. 

Analysis  of  technical  variance  is  probably  the  most 
crucial  type  of  analysis  involved  in  the  management 
process.  It  is  the  one  where  assistance  of  other 
technical  experts  may  be  required.  It  may  also 
require  some  restructuring  of  the  program  merely  to 
determine  the   cause  of   the  variance.   Note  that  any 
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significant  technical  variance  will  always  result  in 
cost  and  schedule  variances.  [REF.  2:p.  4-20J 

Since  no  significant  cost  or  schedule  variances 
have  occurred  in  the  ANGAS  project,  the  most  obvious 
indicators  of  unsatisfactory  technical  progress  are  not 
evident.  LMSC  has  satisfactorily  completed  all  SOW 
items  listed  in  the  first  and  second  phases  of  the 
contract.  To  further  analyze  technical  progress,  the 
ANGAS  project's  system  engineering  will  be  examined. 

MIL-STD  449  defines  system  engineering  as: 

the  application  of  scientific  and  engineering  efforts 
to:  transform  an  operational  need  into  a  description 
of  system  performance  parameters  and  a  system 
configuration  through  the  use  of  an  incentive  process 
of  definitions,  synthesis,  analysis,  design,  test  and 
evaluation;  integrate  related  technical  parameters 
and  assure  compatibility  of  all  physical,  functional 
and  program  interfaces  in  a  manner  which  optimizes 
the  total  system  definition  and  design,;  integrate 
reliability,  maintainability,  safety,  survivability, 
human  and  other  such  factors  into  the  total 
engineering  effort. 

The  contract  SOW   specifies   the   minimum  technical 

requirements   in   the   contract.     Appendix  B  contains 

copies   of   the   first    and    second    contract   phase 

Statements    of     Work.        Examination    of    these 

specifications   reveals   that   they    are   composed   of 

specific    configuration,    design,    performance    and 

reliability   and   maintainability   requirements.     The 

program   manager,   LTCOL   G.  Lasche,  USA,  stressed  that 

the  lessons  learned  in   the  development   of  the  Winkler 

project,  which  used  similar  technology,  was  a  basis  for 

the  technical  requirements  developed  for  ANGAS.    ANGAS 

system    engineering    appears    to    be    a    detailed 

specification  of   valid  technical   goals  developed  from 

past   experience    and   an   overall   knowledge   of   the 

technologies  involved. 

It  is  logical  that   any   analysis   of  technological 

progress    be    based    on   status   reviews   of   system 
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engineering   goals.      The    government's   acquisition 

process  requires  formal  and  informal  design  reviews  to: 

determine  the  adequacy  of  contractor  and  DOD  in- 
house  efforts  towards  achieving  design  goals. 
Participants  should  include  design  attribute 
specialists  in  reliability,  maintainability,  safety, 
and,  particularly,  logistic  supportabi 1 i ty .  Reviews 
should  include  a  preliminary  design  review.  a 
critical  design  review  (CDR),  a  design  certification 
review,  a  functional  configuration  audit f  a  physical 
configuration  audit,  and  a  first-article 
configuration  inspection.  [REF.  2:p.  4-49] 

The  most  recent  formal  review  of   the  ANGAS  project 

was  the  CDR  which  was  presented  to  DARPA  on  July  21  and 

22  of  1987. 

The  CDR,  a  formal  review  of  the  detailed  design  of  a 
configuration  item.  is  performed  by  the  program 
manager  late  in  the  prototype  subphase  when  the 
design  detail  is  essentially  complete,  but  prior  to 
drawing-release  and  fabrication  of  formal  test 
articles.  [REF.  2:p.  4-49] 

The    critical    design   review   demonstrated   that 

technical  and  system  engineering   goals   for   the  ANGAS 

project   were   accomplished   satisfactorily.     The  CDR 

specifically  addressed: 

1.  System  Engineering 

2.  Germanium  Detector  Subsystem  Development  (a 
critical  technical  component  of  the  ANGAS 
system ) 

3.  Shield  Design  (a  critical  technical  component 
of  the  ANGAS  system) 

4.  Mechanical  Engineering  specifications 

5.  Electrical  Subsystem  specifications 

6.  Source  Deployment  Subsystems 

7.  Cooler  Subsystem  specifications  (a  critical 
technical  component  of  the  ANGAS  system) 

8.  Electrical  Ground  Support  Equipment 
specifications 

9.  Monitor  Subsystem  specifications 

10.  Instrument  Test  Plan 

11.  Reliability,  Quality  and  Safety 

12.  Instrument  Development 

In   summary,   ANGAS   technical  goals  and  objectives 
have  been  accurately  detailed  and  specified  as  contract 
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requirements  based  on  past  experience  obtained  from  the 
development  of  systems  using  similar  technologies. 
Formal  and  informal  reviews  and  evaluations  of 
technical  requirements  have  proved  that  the  ANGAS 
project's  technical  goals  are  both  viable  and  capable 
of  being  accomplished. 

This  research  effort  does  not  attempt  to  define  a 
quantitative  measurement  technique  for  evaluating 
technical  variance  and  accomplishment.  An 
understanding  of  the  long-range  R  &  D  planning  steps 
developed  by  Ward  D.  Lowe  and  presented  in  Chapter  III 
is  a  useful  guide  in  establishing  initial  technical 
objectives  for  a  proposed  SOA  extension  project. 

The  prior  development  of  the  Winkler  Program 
reduced  the  risks  and  development  and  provided 
technical  information  on  some  of  the  new  technologies 
used  in  ANGAS.  The  following  brief  description  of  the 
relatively  simple  multi-attribute  utility  method, 
summarized  for  the  Navy  Program  Manager's  Guide, 
provides  an  avenue  for  future  study  of  quantification 
of  technical  goals  and  measurement. 

The  model  addresses  the  management  of  risk  that  is 
inherent  in  decision  making  with  incomplete 
information.  Establishing  SOA  extension  technological 
objectives  and  measurement  specifications  at  the 
initiation  of  a  project  is  often  based  on  incomplete 
information.  The  use  of  this  limited  information  can 
be  optimized  through  a  five  step  process  which  consists 
of: 

1.  Breaking  down  the  tasks  to  be  accomplished  into 
manageable  components  or  attributes. 

2.  Estimating  the  utility  factor,  the  relative 
importance  of  each  component  or  attribute. 

3.  Developing  a  utility  function  or  curve  which 
describes  the  utility  values  as  a  function  of 
some  descriptive  variable  (i.e.,  reliability  in 
terms  of  mean  time  between  failure. ) 
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4.  Estimating  the  risks  associated  with  attaining 
the  utility  values  chosen  for  each  attribute. 

5.  Developing  options  to  avoid  or  overcome 
obstacles  to  success  and  to  compare  alternative 
paths,  solutions,  or  concepts. 


D.  SIGNIFICANT  CHARACTERISTICS  OF  LOCKHEED'S  COST 
ESTIMATING  METHODS  AND  MODELS  THAT  CAN  AID 
GOVERNMENT  COST  ESTIMATING 

Lockheed  sometimes  uses  bottom-up  cost  estimating 
which  is  usually  based  on  historical  data  and  provides 
a  high  degree  of  accuracy.  Cost  estimating  and 
evaluation  within  DOD  should  utilize  historical  costs, 
for  completed  projects  that  are  similar  in  scale  and 
technology,  to  evaluate  projects  under  consideration. 

Use  of  parametric  cost  models,  specifically  the  RCA 
PRICE  Hardware  model  is  used  by  Lockheed  during  the 
early  development  and  concept  exploration  phases  when 
historical  cost  data  does  not  exist.  Parametric  cost 
models,  which  allows  economic  SOA  extension  trade-off 
and  schedule  analysis,  should  be  used  by  DOD  when 
historical  data  does  not  exist  early  in  development 
stages . 

The  Lockheed  internally  developed  STAR  model 
combines  the  characteristics  of  a  similar-to-model  and 
an  associated  model.  This  model,  which  was  described 
in  detail  in  Chapter  II,  is  based  on  a  historical 
database  that  has  been  developed  over  a  20  year  period. 
The  data  base  represents  actual  production  and 
development  cost  experience,  which  can  be  retrieved  by 
line  item  or  box  level  based  on  relatively  simple  input 
characteristics.  Development  of  similar  integrated 
data  base  and  models  within  DOD  or  major  components 
would  significantly  enhance  the  government's  ability  to 
accurately  estimate  SOA  extension  costs. 
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V.  CONCLUSIONS 

The  conclusions  drawn  from  this  research  effort  are 
based  on  the  data  and  analysis  of  the  ANGAS  project. 
These  conclusions  can  be  used  as  a  basis  for  future 
studies  involving  SOA  extension  cost  estimating  methods 
and  case  studies. 

Primary   Research    Question:    How    do   estimated 

development   and    production   costs   compare   with 

actual  costs  for  the  ANGAS  project? 

Development  costs  were  the  primary  costs  estimated 
and  recorded  to  date  due  to  the  early  stage  of 
development  of  the  ANGAS  project.  Total  actual  costs 
were  slightly  lower  than  total  budgeted  costs.  Direct 
labor  and  labor  related  costs  were  higher  than 
budgeted  during  the  first  phase  of  development. 
Variances  did  exist  within  individual  cost  elements.  No 
cost  element  experienced  negative  cost  variances  during 
both  contract  phases  that  were  evaluated  with  the 
exception  of  a  $24  and  $18  cost  overrun  in  overtime 
premiums . 

Subsidiary  Research  Questions: 
1 .   What  methods  and  techniques   were  used   to  estimate 

development  costs? 

LMSC  used  a  bottom-up,  engineering  cost  estimating 
methodology  to  estimate  development  costs.  These 
estimates  were  generated  within  the  Lockheed  Space 
Sciences  Laboratory  based  on  past  experience  of  costs 
incurred  developing  similar  satellite  borne  systems. 
The  bottom-up  estimating  methodology  used  at  LMSC 
provided  detailed  functional  and  cost  element  estimates 
based  on  historical  cost  data  and  specific  contractual 
requirements  as  detailed  in  the  WBS  ,  SOW  and  RFP . 
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2.  How  was  technological  progress  measured  and 
evaluated? 

Technological  progress  was  measured  and  evaluated 
by  monitoring  completion  of  the  specific  configuration, 
design,  performance  and  reliability  and  maintainability 
requirements  stated  in  the  ANGAS  Statements  of  Work 
during  the  formal  and  informal  project  reviews.  The 
program  manager  for  ANGAS  was  able  to  state  technical 
goals  and  objectives  based  on  experience  gained  from 
the  development  of  a  similar  spectrometer  system  that 
used  many  of  the  same  advanced  technologies. 

3.  What  factors  helped  or  hindered  accurate  cost 
estimating  and  accomplishment  of  stated  SOA 
extension  goals? 

Accurate  cost  estimating  was  enhanced  by  the 
existence  of  historical  data  on  similar  type  SOA 
extension  development  costs.  The  division  of  LMSC 
that  was  developing  the  ANGAS  project  was  also 
experienced  in  developing  similar  systems.  ANGAS 
incrementally  extended  the  SOA  based  on  a  smaller  scale 
project  that  combined  many  of  the  same  technologies  but 
was  designed  to  function  in  the  earth's  atmosphere 
instead  of  outer  space.  Historical  cost  data,  technical 
experience  and  incremental  SOA  extension  enhanced  cost 
estimating  accuracy  and  accomplishment  of  technological 
goals . 

Factors  that  hinder  accurate  cost  estimating  and 
accomplishment  of  stated  SOA  extension  goals  arise  from 
the  inherent  risks  of  decision  making  with  incomplete 
information.  The  inability  for  the  government  to 
accurately  specify  project  design  and  specification 
requirements  and  monitor  accomplishment  through  formal 
and  informal  reviews  greatly  hinders  accomplishment  and 
measurement   of   stated   SOA   extension   goals.  Lack  of 
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historical  data,  technical  experience  and  inability  to 
correctly  interpret  contract  requirements  hinders 
contractors  from  correctly  estimating  SOA  extension 
costs . 

Findings ;  There  were  no  significant  variances 
between  total  estimated  and  total  actual  costs  to  date. 
Due  to  the  contractor  uncertainties  associated  with  the 
incremental  funding  used  in  the  project  the  spending 
rate  was  slightly  lower  than  planned. 

Incremental  funding  of  a  DOD  SOA  extension  project 
definite  contract  phases  and  the  opportunity  to  specify 
and  evaluate  technological  objectives  during  each 
phase.  This  type  of  funding  provides  incentives  for  the 
contractor  to  successfully  achieve  technological 
objectives  to  receive  funding  for  later  phases. 
Incremental  funding  provides  leverage  and  flexibility 
to  the  government  in  reducing  cost  overruns  and 
achieving  technological  goals.  The  length  of  time 
required  for  the  development  effort  is  increased  with 
incremental  funding. 

Direct  labor  costs  and  costs  applied  as  a 
percentage  of  direct  labor  costs  were  the  most 
significant  costs  during  the  development  phases.  These 
costs  accounted  for  over  75  percent  of  total  costs  in 
each  phase. 

SOA  extensions  that  incrementally  advance  the  SOA 
reduce  the  risk  of  costs  overruns  and  failure  to 
achieve  technological  goals.  The  historical  cost  data 
and  expertise  obtained  through  the  development  of  the 
smaller  scale  Winkler  project  enabled  DARPA  to 
accurately  detail  system  engineering  specifications  and 
technical  test  parameters. 

The  measurement  and  analysis  of  cost  variances  and 
technological    progress    is    the   primary   gauge   of 
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effectiveness  for  a  research  and  development  effort 
administered  by  a  level  of  effort  contract . Technical 
progress  for  the  ANGAS  project  was  evaluated  as 
satisfactory . 

Currently  available  parametric,  analogous  and 
similar-to  cost  estimating  models  used  by  LMSC  and 
private  industry  could  be  adopted  to  a  greater  degree 
by  the  DOD  to  enhance  SOA  cost  estimating  accuracy. 

Recommendations  for  Further  Study:  Additional  case 
studies  are  needed  to  identify  trends  in  the  cost 
estimating  techniques  and  accuracy  of  SOA  extension 
efforts.  The  computer  based  cost  estimating  models  that 
are  discussed  in  this  research  effort  and  many  other 
computer  based  models  not  addressed  in  this  project 
merit  a  great  deal  of  additional  study.  Identifying 
cost  estimating  methods  that  will  allow  the  most 
efficient  use  of  limited  DOD  funding  by  cost 
effectively  including  new  technologies  in  SOA 
extensions  and  new  system  development  are  vital 
national  objectives. 


55 


APPENDIX  A 
DEFINITIONS 

Bottom-Up  Cost  Estimation  Model:  A  Cost  estimating 
procedure  based  on  detailed  functional  and  cost 
element  estimates  prepared  at  the  lowest  practical 
level  of  task  and  design  detail. 

Budget:  Estimated  costs  of  the  effort  defined  in  the 
work  packages  and  level  of  effort  SOW  items.  These 
costs  should  be  broken  out  at  the  cost  account 
level.  The  estimated  cost  of  the  effort  scheduled 
to  be  accomplished  at  a  point  in  time  within  DQD  is 
termed  "Budgeted  Cost  of  Work  Scheduled"  ( BCWS ) . 

Critical  Design  Review  ( CDR ) :  A  formal  review  of  the 
detailed  design  of  a  configuration  item  is 
performed  by  the  program  manager  late  in  the 
prototype  subphase  when  the  design  detail  is 
essentially  complete  but  prior  to  drawing  release 
and  fabrication  of  formal  test  articles. 

Development  Costs:  Costs  associated  with  research, 
planning  and  design  for  a  new  product  or  process  or 
for  a  significant  improvement  to  an  existing 
product  or  process.  Includes  the  conceptual 
formulation,  design,  and  testing  of  product 
alternatives  and  construction  of  prototypes. 

Intra-Lockheed  Work  Transfer  ( IWT )  Items:  Material 
items  that  are  made  at  Lockheed  and  transferred  at 
cost  to  another  entity  within  the  company. 
Includes  a  separate  breakdown  by  cost. 

Parametric  Modeling:  Cost  estimating  procedure  which 
utilizes  a  model  comprised  of  algorithms  and 
mathematic  equations.  Estimated  costs  are  derived 
from  relationships  of  cost  to  physical  or 
performance  characteristics. 

Purchased  Items:  Includes  material  items  not 
considered  standard  commercial  items  or  raw 
materials . 

Raw  Materials:  Materials  in  a  form  or  state  that 
require  further  processing. 

Standard  Commercial  Items:  Consists  of  items  that  the 
contractor  normally  fabricates,  in  whole  or  in 
part,  and  that  are  generally  stocked  inventory. 

State  of  the  Art  ( SOA )  Extensions:  Used  in  this  thesis 
to  describe  a  new  weapon  system  or  component  that 
has  incorporated  an  advancement  in  technology  in  a 
unique  process  or  for  a  unique  application. 
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WBS  Dictionary:  The  WBS  dictionary  defines  each  WBS 
element  and  describes  the  technical / functional  task 
content  and  responsibilities. 

Work  Break  Down  Structure:  A  product-oriented  family 
tree  composed  of  hardware,  services  and  data  which 
results  from  project  engineering  effort  during  the 
development  and  production  of  a  defense  material 
item  and  which  completely  defines  the  project  / 
program . 
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APPENDIX  B 

STATEMENT  OF  WORK 
FIRST  CONTRACT 

1.  The  work  and  services  to  be  performed  hereunder 
shall  be  subject  to  the  requirements  and  standards 
contained  in  Exhibit  A  and  the  following  paragraph(s). 

2.  The  Contractor  shall  carry  out  a  program  of 
research  in  advanced  gamma-ray  detectors  for  use  in 
space  which  shall  include  the  following  tasks: 

1.  Conduct  gamma-ray  imaging  studies, 

2.  Establish  instrument  GSE  requirements, 

3.  Conduct  preliminary  design  of  germanium  sensor 
and  anticoincidence  shield, 

4.  Prepare  digital  signal  processing  system 
requirements  and  specifications, 

5.  Prepare  cryogenic  cooling  system 
specifications , 

6.  Conduct  preliminary  design  of  collimator, 
wheel,  onboard  hardware,  germanium  sensor 
electronics,  and  antishleld  housing  and 
electronics , 

7.  Prepare  germanium  sensor  procurement 
specifications  and  antishleld  crystal 
procurement  specifications, 

8.  Complete  final  cooler  design  and  performance 
simulation . 
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STATEMENT  OF  WORK 
SECOND  CONTRACT  PHASE 

Under  Modification  P00001 ,  the  Contractor  shall 
conduct  the  final  design  and  sensor / shield  evaluation 
of  a  space  demonstration  experiment  known  as  the  DARPA- 
201  ANGAS  Experiment  aboard  a  free-flyer  satellite 
mission.   The  work  shall  include  the  following  tasks: 

(1)  Finalize  design  of  a  high  sensitivity, 
fine  energy  resolution  imaging  spectrometer,  known  as 
the  ANGAS  instrument,  background  monitors  and  auxiliary 
equipment . 

(2)  Perform  laboratory  evaluations  and  bench 
tests  of  the  Ge  sensor  design  and  of  the  Nal  shield 
system . 

(3)  Conduct  design  studies,  computer 
simulations  and  laboratory  bench  tests  for  performance 
verification  of  the  imagine  techniques  to  be  employed 
on  the  ANGAS  and  finalize  the  design  of  the  collimator 
mask  system  for  the  flight  instrument. 

(4)  Provide  the  electrical,  thermal,  and 
environmental  specifications  and  other  interface 
information  necessary  to  integrate  the  DARPA-201 
payload  aboard  the  satellite. 

(5)  Provide  all  physical.  procedural, 
functional ,  and  safety  data  necessary  to  generate  an 
Experiment  Requirements  Document  (ERD)  consisting  of: 

a)  Interface  Control  Document. 

b)  Ground      Operations      Requirements 
Document . 

(c)   Flight      Operations      Requirements 
Document . 

(6)  Provide  the  inputs  necessary  to  meet  the 
identified  DARPA-201  milestone  schedules.  Departures 
from  these  schedules  must  be  mutually  agreed  to  by 
DARPA  and  the  LPARL. 

(7)  Maintain  an  experiment  quality  program 
during  the  design  and  development  of  the  payload. 

(8)  Design  and  initiate  development  of  the 
ground  support  equipment  ( GSE )  necessary  to  test  and 
integrate  the  DARPA-201  payload  with  the  free-flyer 
satell i te . 

(9)  Provide  DARPA-201  progress  and  status 
information  and  support  the  mission  working  group  and 
status  reviews  as  necessary. 
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STATEMENT  OF  WORK 
PROPOSED  16.5  MILLION  DOLLAR 
COMPLETION  CONTRACT 

The   Lockheed    Missiles   and   Space   Company   will 

provide  to  the  Defense  Advanced  Research  Agency  and  the 

Office   of   Naval   Research   the   necessary   personnel, 

equipment  facilities  and  services   to   conduct   a  space 

demonstration  experiment   known  as   the  DARPA-201  ANGAS 

Experiment  aboard  a  free-flyer   satellite  mission.    In 

the  conduct   of  this   work  the   following  tasks  will  be 

performed : 

1)  Design,  develop,  test  and  calibrate  a  high 
sensitivity,  fine  energy  resolution  imaging 
spectrometer,  known  as  the  ANGAS  instrument  and 
an  auxiliary  background  monitor. 

2)  Conduct  design  studies,  computer  simulations 
and  bench  tests  for  performance  evaluation  of 
the  imaging  techniques  for  intended  employment 
on  the  ANGAS  collimator  mask  wheel. 

3)  Based  on  the  timely  receipt  of  the  electrical, 
thermal  and  environmental  specifications  and 
other  interface  information  from  the 
Government.  provide  the  necessary  information 
to  integrate  the  DARPA-201  payload  aboard  the 
satel lite. 

4)  Provide  all  physical,  procedural,  functional 
and  safety  data  necessary,  to  generate  an  Experiment 
Requirements  Document  (ERD)  consistent  with  the  vehicle 
and  interface  requirements  specified  in  the  STP  From 
1721  (1985).   The  ERD  consists  of 

a)  Interface  Control  Document 

b)  Ground  Operations  Requirements  Document 

c)  Flight  Operations  Requirements  Document 

5  )  Provide  the  inputs  necessary  to  meet  the 
identified  DARPA-201  milestone  schedules. 

6)  Flight  qualify  the  DARPA-201  payload  through 
the  maintenance  of  an  experiment  quality  program  during 
the  design  and  development  of  the  payload.  The 
Principal  Investigator  will  have  full  responsibility 
for  conducting  all  aspects  of  the  quality  program.  The 
quality  program  practices  with  the  ONR . 

7)  Provide  the  ground  support  equipment  ( GSE ) 
necessary  to  test  and  integrate  the  DARPA-201  payload 
with  the  free-flyer  satellite. 

8)  Provide  logistics  and  operation  support  to  the 
DARPA-201  payload  during  definition,  integration  ,  test 
and  on-orbit  operations. 

9)  Provide  DARPA-201  progress  and  status 
information   using   the   Lockheed    R&D   standard 
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management   and   accounting   practices.     Support   the 
mission  working  group  and  status  reviews  as  necessary. 

10)  Based  on  the  timely  receipt  of  government- 
furnished  computer-compatible  agency  data  tapes, 
analyze  the  first  two  years  of  on-orbit  data. 

11)  Provide  flight  performance  reports  as  required. 

12)  Submit  the  scientific  results  acquired  with  the 
DARPA-201  instruments  for  publication  in  the  open 
technical  literature. 
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MAJOR  MILESTONES  FOR  THE 
DARPA-201  ANGAS  EXPERIMENT 

P  r  o  ,j  e  c  t 
Milestones  1985  1986  1987  1988  1989  1990  1991  1992 


Harware  Design 

Procurement 

ICD  Signoff 

Ground  Operations 

required  documentation 

Flight  Operations 

required  documentation 

Hardware  Development 

Instrument  Test 

Hardware  Delivery 

Spacecraft  Prelaunch  Support 

Launch 

Post  Launch  Operations 
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